Skip to content

Drop old quantization flows #3115

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Drop old quantization flows #3115

wants to merge 5 commits into from
+48 −1,825

Conversation

namgyu-youn
Copy link
Contributor

@namgyu-youn namgyu-youn commented Oct 2, 2025 •
edited
Loading

Summary:
This PR drops old quantization flows:

torchao/quantization/subclass.py
torchao/quantization/weight_only.py
torchao/quantization/dynamic_quant.py

Applies corresponding change in other parts for removing old quantization flows:

torchao/quantization/quant_api.py
torchao/quantization/utils.py
torchao/quantization/autoquant.py

# Remove old test cases
test/integration/test_integration.py

# SmoothQuant with old workflow is dropped
# Use torchao/prototype/smoothquant instead with new api
torchao/quantization/smoothquant.py

Test plan: test/integration/test_integration.py

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Oct 2, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/3115

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 6 Pending

As of commit 04ea5f8 with merge base c96f2dd (image):

NEW FAILURE - The following job has failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 2, 2025
@namgyu-youn namgyu-youn mentioned this pull request Oct 2, 2025
Open
6 tasks
@andrewor14 andrewor14 added the topic: improvement Use this tag if this PR is an improvement (doesn't fit into any of the other categories) label Oct 2, 2025
@andrewor14 andrewor14 self-requested a review October 2, 2025 15:58
namgyu-youn
namgyu-youn commented Oct 2, 2025
View reviewed changes
benchmarks/benchmark_aq.py
Copy link
Contributor Author

@namgyu-youn namgyu-youn Oct 2, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about inlining _int8wo_api, _int8da_int8w_api, _int4wo_api ? They are used only once across codebase.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah I think that's fine if they're only used in benchmarks

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also cc @jainapurva, can you take a look at the benchmark changes?

benchmarks/benchmark_aq.py
print("_int8da_int8w_api")

for M, N, K in all_shapes:
_bench_quantized_tensor_subclass_perf(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Temporarily updated to use new APIs 2 times to fix CI, but maybe we can update _bench_quantized_tensor_subclass_perf to compare only original vs. new quantization flows?

andrewor14
andrewor14 reviewed Oct 7, 2025
View reviewed changes
Copy link
Contributor

@andrewor14 andrewor14 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @namgyu-youn, thanks for working on this. I think it looks good overall, but seems like we removed some things outside the scope of #2745, like smoothquant. Can you please add these back?

benchmarks/benchmark_aq.py
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah I think that's fine if they're only used in benchmarks

torchao/quantization/smoothquant.py
@@ -1,266 +0,0 @@
# Copyright (c) Meta Platforms, Inc. and affiliates.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @namgyu-youn I don't think we want to remove smoothquant? This wasn't part of the issue: #2745. Can you add these back?

test/integration/test_integration.py
return wrapper


class SmoothquantUnitTest(unittest.TestCase):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, please add the smoothquant tests back

Copy link
Contributor Author

@namgyu-youn namgyu-youn Oct 7, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In SmoothQuant API side, they (torchao/quantization/smoothquant.py & torchao/quantization/prototype/smoothquant/) are just a two different implementations, so we can revert them.

But in tests side, because we dropped

from torchao.quantization.subclass import (
    Int4WeightOnlyQuantizedLinearWeight,
    Int8DynamicallyQuantizedLinearWeight,
    Int8WeightOnlyQuantizedLinearWeight,
)

,
tests (SmoothquantUnitTest) can't be maintained. So my suggestion is, how about dropping the old SmoothQuant API also in this PR? The new API also resolved #1639 and has better structure I think

torchao/prototype/quantization/autoquant_v2.py Outdated
Int8DynamicallyQuantizedLinearWeight,
Int8WeightOnlyQuantizedLinearWeight,
from torchao.quantization.subclass import (
QuantizedLinearWeightBase,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

didn't we remove this class in this PR? Seems we need to delete this import completely?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, thanks for pointing it out

torchao/quantization/utils.py
return y


def dynamically_quantize_per_channel(x, quant_min, quant_max, target_dtype):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @namgyu-youn I don't think this is part of the scope, can you add this function back? The issue #2745 is only referring to everything in subclass.py, dynamic_quant.py, and weight_only.py

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reverted; it was misunderstanding while checking api structure.

@andrewor14 andrewor14 requested a review from jainapurva October 7, 2025 16:13
@namgyu-youn namgyu-youn requested a review from andrewor14 October 7, 2025 16:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jainapurva jainapurva Awaiting requested review from jainapurva

@andrewor14 andrewor14 Awaiting requested review from andrewor14

Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. topic: improvement Use this tag if this PR is an improvement (doesn't fit into any of the other categories)
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Deprecate and remove subclass.py, dynamic_quant.py, and weight_only.py
2 participants
@namgyu-youn @andrewor14